METHOD AND CIRCUITS FOR ENCODING AN INPUT PATTERN 
USING A NORMALIZER AND A CLASSIFIER 



FIELD OF THE INVENTION 

The present invention relates to data processing and more 
particularly to a method and circuits for encoding an input 
pattern using a normalizer and a classifier in the learning 
phase which significantly reduce the number of required 
prototypes. The present invention also relates to the 
method and circuits to recognize a new (unknown) input 
pattern during the classification phase. 
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BACKGROUND OF THE INVENTION 



Using classifiers in the technical literature, the learning 
and classification (or recognition) tasks require the 
essential step of processing input patterns. During the 
learning phase, this step is performed to define whether or 
not the input pattern needs to be learned in the 
classifier. During the classification phase, for instance, 
in the K Nearest Neighbor (KNN) mode, when a new input 
pattern is presented to the classifier, this step is used 
to find the closest pattern (s) stored in the ANN as 
prototype (s) . Finding the closest stored prototype (s) 
mainly consists to compute the distances between the input 
pattern and the stored prototypes and then, to determine 
the minimum distance (s) among the computed distances. In 
the following description, the preferred classifier is the 
input space mapping algorithm based artificial neural 
network (ANN) described in US patent 5,621,863 assigned to 
IBM Corp that includes innovative elementary processors of 
a new type, referred to as the ZISC neurons, (ZISC is a 
registered trade mark of IBM Corp) . An essential 
characteristic of the ZISC neurons (now commercially 
available in silicon chips) lies in their ability to work 
in parallel, i.e. when an input pattern is presented to the 
ANN, all ZISC neurons compute the distance between the 
input pattern and all the prototypes stored in the neurons 
of the ANN at the same time. One important aspect of these 
algorithms is the distance evaluation relation, referred to 
as the "norm", that is used in the distance evaluation 
process. The choice of this norm is determined by the 
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problem to be solved on the one hand, and on the other hand 
by the knowledge used to solve this problem. In a ZISC 
neuron, the final distance {Dist) between an input pattern A 
and the prototype B stored therein (each having p 
components) is calculated using either the MANHATTAN 
distance (LI norm), i.e. Dw^^i =2|A,-5i| or the MAXIMUM 
distance (Lsup norm), i.e. DistLmp = max.\Ai-Bi\ wherein Ai and 
Bi are the components of rank i (variable i varies from 1 
to d) for the input pattern A and the stored prototype B 
respectively. Other norms exist, for instance the L2 norm 
such as DistLi ^jlliM-Bi? . The L2 norm is said to be 
"Euclidean" while the LI and Lsup norms are examples of 
"non-Euclidean" norms, however, they all imply the 
computation of a difference (A,-fii) for each component in 
the distance evaluation. In the ZISC neuron, the choice 
between the LI or Lsup norm is determined by the value of a 
single bit referred to as the "norm" bit No stored in the 
neuron. Other Euclidean or non-Euclidean norms are known 
for those skilled in the art. 

In many applications based on these input space mapping 
based algorithms, for instance in signal and image 
processing, the main step consists in analyzing an 
essential characteristic of the input patterns. Although a 
determined input pattern and the prototypes stored in the 
ANN can be similar (or even identical) as far as this 
essential characteristic is concerned, the distance 
therebetween may be high due to the other parameters 3 taken 
into consideration in the distance evaluation process. As a 
consequence, it is often necessary to use as many neurons 
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as there are different variations in the input patterns to 
store them as different prototypes. The two examples 
described below will illustrate this major inconvenience. 

Let us first consider two input patterns which are 
electrical signals SI and S2, each representing a voltage 
amplitude varying as a function of time i.e. V=f{t) in a 
range between a minimum value (Vmin) and a maximum value 
(Vmax) . As apparent in FIG. 1, in the interval [to,ti] defined 
by times to and ti, the curves depicting signals SI and S2 
have the same aspect. This common characteristic will be 
referred to hereinbelow as a "shape". They exhibit the same 
variation, because they have the same gradient in absolute 
value and sign, but they are centered around two different 
mean values. The mean value is thus the parameter which 
distinguishes the two signals SI and S2 in the time 
interval in consideration. In the instant case, let us 
assume the voltage range (from Vmin to Vmax) is coded on 8 
bits with 16 points of sampling. In this time interval, 
assuming Vmin=0 and Vmax=255, signals SI and S2 are 
respectively represented by the values given in tables Tl 
and T2 after sampling and coding. In the FIG. 1 case, 
because the amplitude values of signals SI and S2 are 
different, two neurons having at least 16 components are 
required to store signals SI and S2 as two different 
prototypes in the learning phase even if their variations 
look similar on the totality of the time interval. On the 
other hand, in the classification phase, when an input 
pattern is presented to the ANN for classification, e.g. 
signal S3 (corresponding values are given in table T3) 
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placed between signals SI and S2 and having the same 
variation in said time interval but a different mean value, 
it could be not recognized as close (i.e. similar) to 
either signal SI or 32. 

Now, let us consider the case where the input patterns are 
two sub- images II and 12 globally having a similar aspect 
as shown in FIG. 2, Images II and 12, that are comprised of 
three stripes of twelve pixels (for a total of 36 pixels) , 
slightly distinguishes one from another, in fact, they only 
differ in terms of global brightness. In the instant case, 
the brightness of each pixel of sub-images II and 12 is 
coded on 8 bits to have a coding values ranging from 0 
(black) to 255 (white) with all the permitted gray levels 
therebetween. The result of this coding is illustrated by 
tables T'l and T'2. As apparent in tables T'l and T'2, 
sub- images II and 12 have the same brightness gradient, 
giving them said similar aspect mentioned above and this 
common characteristic will still be referred to hereinbelow 
as a shape " . As described above, because the brightness 
values of the pixels of sub- images II and 12 are different, 
two neurons having the capability to memorize 36 components 
(corresponding to the 36 coding values of either table 
which characterize the brightness of a sxib- image) are 
required to store these two sub- images as prototypes during 
the learning phase even if they have the same shape. 

In conclusion, it is thus often necessary to use as many 
neurons as there are possible variations of a same shape. 
These variations being defined by at least one parameter 
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(in the two above described examples in terms of mean value 
or of brightness respectively) of a same shape. 

Of course, the problem described above by reference to 
FIGS. 1 and 2 can be generalized to input patterns having 
dimensions greater than one (see FIG. 1) or two (see FIG. 
2) . 

The number of neurons is not unlimited in conventional 
ANNS, so that, when several neurons are required to 
memorize a same shape, it results a significant waste of 
the capacity of the ANN to store prototypes and finally, 
less different shapes can be stored therein. Finally, using 
several neurons to memorize a same shape decreases the ANN 
response quality. 

Therefore, requiring as many stored prototypes, i.e. 
neurons, as there are possible variations of a same shape 
is thus a major inconvenience in conventional ANNs. 

SUMMARY OF THE INVENTION 

It is therefore a primary object of the present invention 
to provide a method and circuits for encoding an input 
pattern using a normalizer and a classifier, such as an 
input space mapping algorithm based artificial neural 
network (ANN) , in the learning phase and the classification 
phase . 

It is another object of the present invention to provide a 
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method for encoding an input pattern in the learning phase 
using a normalizer and a classifier that requires only one 
neuron to store a normalized pattern that represents a 
plurality of input patterns having an essential 
characteristic in common but which only differ on at least 
one parameter related thereto. 

It is another object of the present invention to provide a 
method and circuits for encoding an input pattern in the 
learning phase using a normalizer and a classifier wherein 
each input pattern of said plurality is identified by a 
couple of values. 

It is another object of the present invention to provide a 
method and circuits for encoding an input pattern in the 
learning phase using a normalizer and a classifier wherein 
said normalized input pattern and couple of values 
represent the encoded pattern. 

It is still another object of the present invention to 
provide a method and circuits for encoding an input pattern 
in the learning phase using a normalizer and a classifier 
that significantly reduce the number of required 
prototypes, improving thereby the classifier response 
quality. 

It is still another further object of the present invention 
to provide a method and circuits for recognizing an 
(unknown) input pattern that has been previously learned as 
a prototype in the codebook memory of a classifier by using 
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a normal izer and this codebook memory. 

It is still another further object of the present invention 
to provide a method and circuits for decoding an encoded 
pattern that has been previously learned in a classifier 
during the classification phase using a denorraalizer and 
the codebook memory thereof. 

Let us consider a plurality of input patterns that have an 
essential characteristic in common but differing on at 
least one parameter related thereto (which thus varies from 
one input pattern to another) . According to the method and 
circuits for encoding an input pattern of the present 
invention, during the learning phase, each input pattern is 
first normalized in a normalizer, before it is presented to 
a classifier, typically an ANN. If not recognized, it is 
learned, i.e. the normalized pattern is stored in a neuron 
of the ANN as a prototype with its category assigned to it 
by the user. As a result, all the input patterns that have 
an essential characteristic left invariant by the 
normalizer (shape) but differing on the value of said at 
least one parameter (main parameter) , are thus represented 
by the same normalized pattern. To that end, the normalizer 
computes an element, referred to as the main factor, which 
allows to extract the normalized pattern from the input 
pattern. Vice versa, a denormalizer is used to retrieve the 
input pattern from the normalized pattern. As a 
consequence, any input pattern is simply and fully 
identified by a couple of values: the category (assigned to 
the input pattern by the user) and the main factor. This 
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couple of values (identification data) and the normalized 
pattern represent the encoded pattern according to the 
present invention. 

The normalization step is performed according to a specific 
mathematical transformation (e.g. a shift, a rotation, ... 
) which depends upon the application in consideration. For 
instance, let us assume that said input patterns are 
identical sinewave signals (in amplitude and frequency) 
having different mean values and that the main parameter is 
the mean value thereof. The normalized signal is either 
input signal shifted at the center value (selected as the 
reference for that parameter) of the range coding the mean 
values of all these sinewave signals. Any input signal is 
then represented by the normalized signal and it is only 
identified by its category and its main factor, i.e. the 
offset between its mean value and said reference value. 

During the classification phase, when a new (unknown) input 
pattern is to be recognized, it is first normalized in a 
normal izer by computing the main factor the same way as 
described above. Then, the normalized pattern is presented 
to the ANN. If it has been previously learned, the ANN 
supplies its category. This category and the main factor 
are identification data that identify said new input 
pattern and which can be subsequently combined with the 
normalized pattern to retrieve the original pattern that 
was previously learned close to the new input pattern. 

The method allows a significant reduction of the number of 
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required prototypes, and thus of neurons, improving thereby 
the ANN response quality. 

The novel features believed to be characteristic of this 
invention are set forth in the appended claims. The 
invention itself, however, as well as other objects and 
advantages thereof, may be best understood by reference to 
the following detailed description of an illustrated 
preferred embodiment to be read in conjunction with the 
accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 shows three curves representing electrical signals 
SI, 82 and S3 having the same voltage amplitude variation 
in a determined time interval [to,ti] as illustrated by the 
corresponding values depicted in tables Tl, T2 and T3 . 

FIG. 2 shows two sub-images II and 12 having the same 
brightness variations as illustrated by the corresponding 
values depicted in tables T'l and T'2. 

FIG. 3 shows curves representing signals SI and S2 of FIG. 
1 and signal S which is obtained after normalization of 
either signal SI or S2 and the corresponding values 
depicted in table T according to the present invention. 

FIG. 4 shows the sub- image I which is obtained after 
normalization of either sub-image II or 12 of PIG. 2 and 
the corresponding values depicted in table T' still 
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according to the present invention. 

FIG. 5 shows curves representing electrical signals S4 and 
S' wherein the mean value of signal S' obtained after 
normalization of signal S4 cannot be set at the center of 
the voltage range for overflow reasons • 

FIG. 6 shows a more general case wherein four sub- images II 
to 14 can be represented by only one normalized sub-image 
I' irrespective of variations in brightness, contrast and 
orientation still according to the present invention. 

FIG. 7 represents the encoding unit adapted to encode the 
input patterns according to the method of the present 
invention. 

FIGS. 8a and 8b respectively represent the encoding unit to 
recognize a new (unknown) input pattern for its 
identification and the decoding unit that is further 
required if the original input pattern that has been 
previously learned needs to be retrieved still according to 
the method of the present invention. 

FIG. 9 illustrates an example of the present invention when 
it is used for image compression/ decompression based on 
Vector Quantization (VQ) to reduce the number of required 
prototypes stored in an ANN and thus to improve its 
response quality. 

FIG. 10 is a block diagram of one embodiment of a typical 
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apparatus incorporating features of the present invention 
that may be used to practice the present ivnention. 

DESCRIPTION OF A PREFERRED EMBODIMENT 

Using the above notations, an essential characteristic that 
is common to a plurality of input patterns for a particular 
application is referred to as a ''shape" . The differences 
between the input patterns are measured through different 
values of at least one parameter referred to as the ''main 
parameter''. In that meaning, a main parameter thus modifies 
the input pattern in some extent but not this essential 
characteristic for the application in consideration. 

For the sake of illustration, the encoding method of the 
present invention will be described by reference to the 
processing of signals SI, S2 and sub- images II, 12 shown in 
FIGS. 1 and 2 respectively. 

As it can be seen in FIG. 1, input signals SI and S2 have 
the same voltage amplitude variation in the time interval 
[toJilr so that only one signal S could represent this common 
characteristic as a shape. In order to optimize the coding 
dynamic, preferably, signal S should be centered as close 
as possible of the coding range center value, i.e. 
(Vmax- Vmin)/2 . Still assuming that ymin = 0 and ymax = 255 for the 
sake of simplicity, signal S should be centered at V=127. 
The voltage mean values of SI and S2 signals (given by the 
sum of all the coding values divided by the total number of 
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sampling points would be Vi=63 and V2 = 222 respectively. 
Thus, with respect to signal signal SI must be shifted 
by an offset value equal to Offseti=V-V'i = m-63 = 64 and 
signal S2 must be shifted by an offset value equal to 
Offset! = V-V^=m- 222 = -95 . 

FIG. 3 shows the signal S that is obtained as a result of 
either shift and the corresponding voltage amplitude values 
(centered around T'=(Vmax"-' Vmin)/2= 127 with the above 
assumptions) are depicted in table T. Signal S is said 
''normalized" and aims to represent all signals having the 
same voltage amplitude variation in the said time interval 
[toJi]. In the instant case, the normalization, performed in 
a normalizer, thus consists to compute the respective 
offset for each signal SI and S2 and then to set the 
normalized signal S, which is thus either signal SI or S2 
shifted to a mean value set as close as possible of the 
center (selected here as the reference value for the 
normalizer) of the mean range coding values. As apparent in 
FIG. 3, normalized signal S which is centered around the 
(Vmax- Vmin)/2 value, represents either signal SI or S2 with an 
offset between their respective mean values equal to Offset\ 
or Offset!. These offset values that are computed in the 
normalizer, are referred to hereinbelow as the ''main 
factors" of signals SI and S2 . 

Let us now consider input signals SI, S2, ... that are 
sequentially applied for the first time to an unit 
comprised of a normalizer and an AHN that are serially 



FR919990123 



13 



connected. Signal SI is noinnalized and appears as 
normalized signal S with a main factor equal to Offsets Now, 
normalized signal S is presented to the ANN, and if not 
recognized by the ANN, it is learned and stored in a neuron 
as a new prototype. Then, when signal S2 is presented, the 
normalizer performs the same operation, but in this case, 
the normalized signal S that is produced, is recognized by 
the ANN and therefore is not stored therein as a different 
prototype. It just remains its main factor Offset2 that is 
computed by the normalizer. More generally, if we consider 
different types of signals (e.g. sinewave, pulse, sawtooth 
and the like) which belong to different categories 
according to values assigned by the user (in the usual 
meaning used in the conventional ANN field) , any input 
signal processed as described above, that has been learned, 
will be represented by a normalized signal, with its main 
factor and category values associated thereto. The main 
factor and the category are the identification data 
mentioned above, in other words, its signature. Therefore, 
during the learning phase, instead of storing signals 81 
and S2 in two different neurons of the ANN as standard, 
only one neuron is used to store normalized signal S as a 
prototype. Either input signal SI or S2 is represented by 
said normalized signal S (stored in a neuron) , its category 
and its main factor (offset) . This combination will be 
referred to hereinbelow as the encoded input signal. 

During the classification phase, the same normalization is 
performed on the new (unknown) input signal before it is 
compared with all the prototypes stored in the ANN for 
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distance evaluation. If recognized by a neuron, e.g. the 
neuron storing normalized signal S, the ANN will output its 
category. This category and the main factor (provided by 
the normalizer) represent said identification data that are 
sufficient to identify the new input signal in 
consideration. However, it is possible to go further and 
retrieve an input signal that is close to the original 
input signal (SI, S2, . . . ) . To that end, the category is 
applied to the ANN storing the normalized signals, so that, 
if it is recognized, the corresponding normalized signal, 
e.g. normalized signal S is output from the ANN. It is to 
be noted that only the storage part of the ANN (referred to 
as the codebook memory) is used in this case, but not the 
calculation part thereof. The normalized signal S is then 
applied to a denormalizer which receives the main factor as 
an input data and performs the reverse operation to 
retrieve an input signal that is close to the original 
input signal . 

Tables T'l and T'2 depicted in FIG. 2 shows the brightness 
values of the pixels forming sub- images II and 12, which in 
turn allow to determine the brightness mean value and then 
to compute the required offsets that are necessary for 
their normalization. Once, the brightness values of 
sub- images II and 12 have been shifted by these offsets, 
the shape is then represented by normalized sub- image I and 
its corresponding table labeled T' in FIG. 4. During the 
learning phase, the normalized sub- image I is stored in one 
neuron of the ANN as a prototype. The computed offsets are 
the main factors which in combination with their associated 
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category (given by the user) still form the said 
identification data mentioned above for either sub-image II 
or 12. During the classification phase, the same 
normalization is performed on the new (unknown) sub- image 
before the normalized sub- image is compared with the 
prototypes stored in the ANN to detennine its category if 
recognized (i.e. if it has been previously learned). This 
category and the main factor (computed by the normalizer) 
represent the identification data that are sufficient to 
identify the new input sub- image in cons iderac ion. The next 
steps of retrieving an sub- image as close as possible of 
the original input sub- image that has been learned are the 
same as described above by reference to normalized signal 
S. 

Therefore, according to the method of the present 
invention, each input pattern is presented to an encoding 
unit comprised of a normalizer and an ANN that are serially 
connected • The main factor is computed and the input 
pattern is normalized before it is presented to the ANN. As 
a result, only normalized patterns are stored as prototypes 
in the neurons of the ANN during the learning phase (if 
they need to be learned) . The normalization thus consists 
to compute the main factor and then to set the normalized 
pattern. In the two above described examples, the main 
parameter is a voltage and a brightness respectively, and 
in this case, the normalization consists to compute the 
respective offsets with the center value of the mean value 
coding range and then to perform a shift. More generally, 
one could envision that each input pattern is normalized 
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using a specific mathematical transformation different from 
a shift, for instance a rotation. After encoding, each 
input pattern is thus finally characterized by its 
normalized pattern and identification data attached 
thereto: its category and its main factor (an offset value 
in both cases) . During the classification phase, the same 
normalization process is performed on the new (unknown) 
input pattern before it is compared with the prototypes 
stored in the ANN for distance evaluation. To that end, the 
same encoding unit is used. If the normalized pattern is 
recognized, the ANN supplies its category. The category and 
the main factor computed by the normalizer are 
identification data that are sufficient to subsequently 
retrieve an input pattern close to the original input 
pattern that has been learned. 

In the example illustrated by reference to FIG. 3, the 
normalization consists in shifting the input signals SI and 
S2 (taking the mean value coding range center as the 
reference) without taking into account a possible overflow. 
However, in some cases it may happen that some values of 
the normalized signal may be out of the coding range, i.e. 
lower than the minimum or greater than maximum value of the 
components. A ZISC ANN is comprised of a plurality of 
neurons, each neuron has a limited number of components, 
each component being coded between determined minimum and 
maximum values. Typically, each ZISC neuron has 64 
components, each being coded on 8 bits (i.e. 256 values 
between 0 and 255) . Any overflow would obviously contribute 
to decrease the ANN response quality. FIG. 5 illustrates a 
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pulse signal S4 which cannot be set at the center (half 
value) of the mean value coding range, i.e, (Vmax - Vmin)/2, as 
described above for normalization. As apparent in FIG. 5, 
if signal S4 mean value is set to half of the coding range, 
the top part of the signal will be clamped which is not 
acceptable • In this case, the normalized signal represented 
by S' is obtained by shifting signal S4 with an offset 
equal to Offset4 so that its mean value is not equal to the 
half value of the coding -range. More generally, it must be 
checked that the normalized signal is within the (ymax,V'min) 
range. To overcome this drawback, the minimum and maximum 
values of an input pattern are used during the offset 
computation. As a result, the normalization now thus 
consists in shifting an input pattern as close as possible 
to the center of the mean value coding range but in such a 
way that the minimum and maximum component values are not 
out of the minimum and maximum coding range values. The 
component values of the normalized pattern to be learned 
(or classified) are evaluated using the following equations 
wherein: 

Aij is the value of the component i of the input pattern 

j; 

A)y is the value of the component i of the normalized 
pattern j ; 

Meanj is the mean value of the pattern j ; 

Mirtj is the minimum value among all the components of the 
input pattern j ; 

Maxj is the maximum value among all the components of the 
pattern j ; 
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Offset] is the value used to shift the pattern j in order to 
set its mean value as close as possible to the mean value 
coding range; and, 

Max is the maximum value that can be reached by a 
component (for the sake of simplicity, we assume that the 
minimum value that can be reached by a component is zero. 
The equations are: 

n 

(1) Meanj^THiAij 

(2) Mm/=min(A») 

(3) Max/ =max (A ») 

i=\..n 

( 4 ) Offset j = \Max - Mearij 

If Offsetj>0, then Offsetj = imn(Offsetj,Max-Maxj) 
If Offsetj<0, then Offsetj = m30iiOffsetj,-Minj) 

and finally, the component values of the normalized pattern 
are such as : 

A'ij=Aij + Offset j 

A normalizer adequate to perform this transformation based 
on equations (1) to (4) to compute the main factor {Offset j) 
that is necessary to set the normalized pattern, e.g. by 
shifting the input pattern of that value, can be designed 
or integrated in a programmable logic device (PLA) without 
any difficulty for the skilled professional. 

For sake of clarity, the above described normalization 
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process is only based on a shifting operation. However, it 
will be obvious that other mathematical transformations may 
be used as well. 

In that respect, FIG. 6 shows four different sub- images II 
to 14 having a similar aspect but which differ on three 
main parameters : brightness, contrast and orientation. An 
offset used to shift the sub- image for the brightness, an 
adapted gain for the contrast and a rotation angle to 
adjust the orientation, represent the three main factors 
for this normalization step. Finally, these four sub- images 
are represented by normalized sub-image I' stored in the 
ANN as a single prototype. The normalized sub-image I' is 
thus the invariant part of sub- images II to 14 with regards 
to these three main parameters. During the classification 
phase, each of the four sub- image II to 14 will be 
recognized as described by normalized sub-image I'. 

FIG. 7 shows the encoder unit to illustrate the method of 
encoding the input patterns during the learning phase 
according to the present invention. Now turning to FIG. 7, 
the encoding unit 700 is comprised of a normalizer 701 and 
a classifier 702 such as the ZISC ANN mentioned above that 
are connected in series. An input pattern is applied to the 
normalizer 701 which computes the main factor F for the 
main parameter in consideration (for instance an offset 
with a reference value thereof) and extracts the normalized 
pattern (i.e. the shape) . The normalized pattern is 
presented to the classifier 702 with its category C 
assigned by the user as standard. If not recognized, the 
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normalized pattern is learned, i.e, it is stored in a 
neuron of the classifier 702 as a prototype with its 
category C associated thereto. The category C and the main 
factor F are the identification data mentioned above, 
which, in combination with the normalized pattern represent 
the encoded pattern according to the present invention. 

During the classification phase, the normalized patterns 
stored in the codebook of the classifier are used to find 
the closest prototype (s) matching with the new (unknown) 
input pattern and then, if so required, an additional step 
is performed to retrieve a pattern close to said new input 
pattern as it will be now described by reference to FIGS, 
8a and 8b. 

Turning to FIG. 8a, the new (unknown) input pattern is 
applied to an encoding unit 700' comprised of a normalizer 
701' and a classifier 702' that are serially connected 
(units 700 and 700' are quite identical from an hardware 
point of view except in that unit 700' does not need to 
receive the category signal C. Normalizer 701 computes the 
main factor F and extracts the normalized pattern as 
described above by reference to FIG. 7. The normalized 
pattern is then presented to the classifier 702. If the 
normalized pattern is recognized, the latter outputs its 
category C. The combination of the category C and the main 
factor F are the identification data mentioned above that 
suffice to identify the new input pattern. If it is 
required to retrieve the input pattern corresponding to 
these identification data, a decoding unit 800, comprised 
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of the codebook memory 801 of classifier 702 and a 
denormalizer 802 that are connected in series as apparent 
in FIG. 8b, has to be used. 

Now turning to FIG. 8b, the category C is applied to the 
codebook memory 801 as an address, normalized pattern 
corresponding to that category is output and applied to the 
denormalizer 802 which also receives the main factor F. A 
pattern that has been learned, i.e. close to the input 
pattern applied to unit 700', is then reconstructed in the 
denormalizer 802 by the reverse mathematical transformation 
and is output therefrom. The identification data (a couple 
of values C and F) for each new input pattern can be stored 
in a RAM memory 803 for convenience. In the RAM memory 803, 
said identification data form a table with two entries that 
allows an easy and immediate identification of any input 
pattern that has been learned. 

The method of the present invention may be efficiently used 
in conjunction with the Vector Quantization (VQ) , a 
technique widely used for image /video compression. The 
advantage of VQ compression is that it needs only standard 
hardware for compression (for example an ANN constructed 
with ZISC neurons) and real-time software for decompression 
(e.g. based on the principle described below) . In 
particular, this technique is well adapted for image/video 
compression on Internet where it could be possible to use 
servers based on ANNs for compression and user's personal 
computer for decompression. 
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The standard VQ compression algorithm is based on 
sub-images analysis. A codebook is made up (statically or 
dynamically) to associate an index to each sub- image of an 
image. For the sake of simplicity, it is assumed in the 
following description that gray level images are used. Each 
sub- image of an image to be compressed may be referenced by 
the index of the closest sub- image stored in the codebook 
memory. Thus, to transmit an image, a set of indexes is 
transmitted instead of all the pixels of the image. For 
example, considering a 1024x768 pixel image (XGA) and 8x8 
pixel sub-images, it is necessary to transmit =12,288 
indexes instead of 1024jc768 =786,432 pixels. Now, considering 
that each pixel is coded with a byte and each index is 
coded with 14 bits, the compression ratio is -^^^xj^ =36.51 . 
However, although there is a good compression ratio, the 
main problem lies in the number of sub- images that is 
required to totally fill the codebook, i.e. 2^^ This number 
must be large enough to obtain a good quality response. A 
blocking effect due to an insufficient number of sub- images 
stored in the codebook often appears when using this 
compression technique. 

Using the method of the present invention, the quality of 
the decompressed image can be increased while decreasing 
the number of sub- images stored in the codebook memory of 
the classifier. When making up the codebook, the input 
patterns are normalized before they are stored as 
prototypes in the classifier (e.g. a ZISC ANN) . 

FIGS. 9a and 9b illustrate the compression and 
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decompression techniques based on the methods of encoding 
and decoding described above by reference to FIGS. 8a and 
8b respectively. During the compression process, the offset 
(or bias) of each sub-image 901-1, 901-2, extracted 
from the image 900 to be compressed is nonnalized in block 
902. Then, using the classifier 903 of FIG. 9a, the closest 
sub- image of the codebook, memorized as a prototype, is 
found. Thus, each normalized sub-image of image 900 is 
characterized by the index of the closest sub- image stored 
in the codebook. After having processed all uhe sub- images, 
the encoded image, i.e. the compressed image, consists in a 
set of characterizing indexes and offsets. 

Now, turning to the decompression process, the 
characterizing indexes are used to extract the closest 
normalized sub- image stored in the codebook memory 904 of 
classifier 903 and this sub-image is denormalized using its 
offset in denormalizer 905 to produce a decompressed 
sub-image 901' -1, 901' -2, ... . The encoded image is thus 
decompressed by processing the set of characterizing 
indexes/offsets to produce the decompressed image 900' • 

Using the above example, still considering a 1024x768 pixel 
image and 8x8 pixel sub- images, it is necessary to transmit 
12,288 indexes and 12,288 offsets instead of 786,432 pixels. 
Now, considering that each pixel is coded with a byte, that 
each index is coded with 9 bits and each offset is coded 
with 5 bits (still for a total of 14 bits) , the compression 
ratio remains equal to 36.57 = ^^^jCg^, but the number of 
sub- images that is now required in the codebook memory is 
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reduced to 2' instead of 2'^ with the standard VQ technique. 

Dealing with color images in which each pixel is 
represented with three values (generally RGB or YUV) does 
not really affect this method. A first approach consists in 
using sub- images having a third dimension to code the color 
component, e.g. red, green and blue (R, G and B) . For 
example, instead of using 64 component siib- images (8x8 
pixels), 192 components are used (8x8x3). However, due to 
the larger component number, more prototypes are required 
to be mapped with the same precision, so that the image 
compression quality may be disappointing. A second solution 
consists in considering a color image as three independent 
images, each representing a color component, e.g. red, 
green and blue (R, G and B) , and then processing 
independently these three images during compression and 
decompression phases. The codebooks used for each type of 
image, e.g. R, G and B, may or not be the same. 

The present invention may also include software and 
computer programs incorporating the process steps and 
instructions described above that are executed in different 
computers. Fig. 10 is a block diagram of one embodiment of 
a typical apparatus incorporating features of the present 
invention that may be used to practice the present 
invention. As shown, a computer system 100 may be linked to 
another computer system 102, such that the computers 100 
and 102 are capable of sending information to each other 
and receiving information from each other. In one 
embodiment, computer system 102 could include a server 
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computer adapted to communicate with a network, such as for 
example, the Internet. Computer systems 100 and 102 can be 
linked together in any conventional manner including a 
modem, hard wire connection, or fiber optic link. 
Generally, information can be made available to both 
computer systems 100 and 102 using a communication protocol 
typically sent over a communication channel 108, or through 
a dial-up connection on ISDN line. Computers 100 and 102 
are generally adapted to utilize program storage devices 
einbodying machine readable program source code which is 
adapted to cause the computers 100 and 102 to perform the 
method steps of the present invention. The program storage 
devices incorporating features of the present invention may 
be devised, made and used as a component of a machine 
utilizing optics, magnetic properties and/or electronics to 
perform the procedures and methods of the present 
invention. In alternate embodiments, the program storage 
devices may include magnetic media such as a diskette or 
computer hard drive, which is readable and executable by a 
computer. In other alternate embodiments, the program 
storage devices could include optical disks, 
read-only^nemory (^^RON") floppy disks and semiconductor 
materials and chips. 

Computer systems 100 and 102 may also include a 
microprocessor for executing stored programs. Computer 102 
may include a data storage device 104 on its program 
storage device for the storage of information and data. The 
computer program or software incorporating the processes 
and method steps incorporating features of the present 



FR919990123 



26 



invention may be stored in one or more computers 100 and 
102 on an otherwise conventional program storage device. In 
one embodiment, computers 100 and 102 may include a user 
interface 106, and a display interface 108 from which 
features of the present invention can be accessed. The user 
interface 106 and the display interface 108 can be adapted 
to allow the input of queries and commands to the system, 
as well as present the results of the commands and queries. 

While the invention has been particularly described with 
respect to a preferred embodiment thereof it should be 
understood by one skilled in the art that the foregoing and 
other changes in form and details may be made therein 
without departing from the spirit and scope of the 
invention. 
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